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Strategic Academic Activities: Bring Them Together With A Program Portfolio 



Abstract: 

With continued waves of change in the management of academic programs, new ideas are met 
with resistance, apathy, and sometimes hostility simply because they represent more c ange 
After initiating program review, a new strategic plan, learning goals, and outcomes assessment, 
we decided to extend the data provided to manage departments and colleges. Rather than mitiate 
yet another new project, meeting the need for more in information was accomplished by creating 
a program portfolio. This portfolio provided a means for integrating many of the previo^ 
strategic initiatives. As the steps are traced, lessons learned and next steps are identified and 

discussed. 



Step I. Institutional Changes and Identifying The Need for Information 

DePaul's recent history has been one of rather marked institutional change that has left some 

residual fhistrations. In the 1996-97 self study prepared for the North Central Association 

accreditation visit, DePaul listed its Institutional Environment Goal: 

“To create an institutional environment that supports a learning community of high 
quality and an efficient and effective administrative infi'astructure. Changes in the 
nature of knowledge and learning and changing expectations for the way DePaul 
relates to society and the civic community will affect institutional structures, methods 
of operation and standards of performance. Significant growth in student, faculty and 
staff populations, increased demands on internal systems and recent changes in senior 
personnel are more than enough to challenge any management structure.” 

(p55, DePaul University: The second hundred years. Self-study report. Executive 
Summary, North Central Association, November 1996) 

In the past four years, DePaul has launched a comprehensive Strategic Plan for growth; designed 
and implemented a new university-wide system of Academic Program Review; created an Office 
of Teaching Learning and Assessment; implemented a new General Education Program; and 
implemented several additional initiatives under a Title III Strengthening Institutions Grant to 
improve undergraduate retention and advising. Additionally, in the last year, the university has 
begim to migrate all of its administrative information systems to PeopleSoft. Each project has 
brought about a great deal of change by introducing numerous program-level and university- 
wide initiatives to the faculty, staff, and to the administrative managers and decision-makers at 
the university. At the same time, these projects have also generated the need for more 
information about the university's progress and the need to improve the management of our 
institutional learning process. The following summaries of each major project describe the 
types of initiatives being implemented and the types of information needs they generate. 

Strategic Plan. In the 1998-99 academic year the university launched a comprehensive 
strategic plan. The plan, "Vision 2006," calls for substantial growth over the next seven years, 
an additional 6,000 students, new facilities, new technology, new administrative systems, new 
revenue, new faculty, and new programs. It has a major goal for the improvement of traditional 
undergraduate education. It has a major goal for the strengthening of our work with the part-time 
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urban student often seeking professional education. It has a major goal for growing the 
contribution we make to the community through our service centers and projects. A final goal is 
to remain solvent while we accomplish the first three goals. Within its goals, several initiatives 
have been launched to support these goals, each headed by a task force charged to develop, 
implement, and evaluate the initiatives' progress. Taskforces include topics such as diversity, 
technology, financial accessibility, and metropolitan presence. In addition, we make an annual 
report to our Board on the progress we are making on these goals. As the university moves 
forward with its strategic plan, it is essential to design ways to measure the success of these ' 
initiatives and to bring together a wide array of information about their progress at the imiversity 
and program level. 

Title HI Strengthening Institutions Grant. Awarded in 1 995, the Title IB Grant supports 

three related activities designed to assess and strengthen the quality of imdergraduate academic 
programs, improve student retention and graduation rates, and decrease attrition, particularly of 
freshmen. These initiatives are the restructuring of our general core education requirements, the 
development of an ability to increase the quality of the advising provided students, and the 
improvement of the quality of teaching and learning particularly through the assessment of 
learning outcomes. Each activity has generated increased administrative interest for learning 
more about how we define and measure student success. Also faculty have become increasingly 
engaged in program-level efforts to improve their teaching and assess student learning and are 
asking for information about their students' achievement and academic progress. Likewise, as 
the new general education program is rolled out and improvements to advising made, there is a 
need for more information about their impact on imdergraduate student success (see 
http://www.depaul.edu/~titleiii). 




Teaching, Learning, and Assessment To build on the work being done under the auspices of 
the Title III grant, the university has instituted several infrastructure changes, including the 
creation of a Faculty Advisory Board on Teaching, Learning, and Assessment and the position of 
Director for Teaching, Learning, and Assessment at the university level. The university has also 
just recently approved an Office of Teaching, Learning and Assessment with the mission and 
primary goals being 1) to promote approaches to teaching and assessment that improve student 
learning; 2) to support scholarly research that advances our understanding of student learning 
across the University; 3) to facilitate faculty sharing of teaching, learning, and assessment issues; 
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and 4) to examine institutional policies and structures for their impact on teaching and learning. 
This Office supports events for faculty to discuss teaching, assessment, and student learning; 
provides a clearinghouse of information, expertise, and research on ways to improve teaching, 
student learning, and assessment; and facilitates the sharing of information among faculty from 
across the university on teaching learning and assessment (for more information see 
http://www.depaul.edu/~titleiii/assessment.htm V 

Academic Program Review. In response to the North Central Association's reconunendations to 
DePaul, the university designed and implemented in 1997-1998 a new system of university-wide 
academic program review to promote the continuous quality improvement of academic programs 
and of the larger university. The new system, designed and led by a university-wide faculty 
committee, reviews all academic programs on a rotating basis over a seven-year period. In 
addition to the traditional academic programs, the academic centers and institutes will also be 
reviewed, which will create special information needs to assist in assessing their performance. 
During the review process, a faculty team from each academic department or program (academic 
‘unit’), conducts a self-study to assess the unit’s strengths and limitations within the larger 
context of the university. This self-study is shared with the faculty on the university review 
committee, the dean, and the Executive Vice President for Academic Affairs, who, together with 
the unit’s faculty, establish and agree upon an accountable plan of action for strengthening the 
unit’s academic programs. Although reviews are scheduled for a unit once every seven years, 
the intent of this process is to support each academic unit in developing and maintaining its own 
continuous, naturally-embedded system of review. What is learned about academic quality can 
then inform planning and decision-making at the unit, college, and university levels. 

At the program-level. Academic Program Review has generated an immediate need for 
program-specific data on students, courses, and faculty-data pertaining to academic quality that 
can be routinely accessible and available to those who are reviewing the programs. At the 
university-level, academic program review has created the opportunity for fatuity to provide 
university leaders with information about the activities, potential, and needs of their individual 
programs (see http://pres.depaul.edu/aprc). 

Local Motivation: One of the major factors has been the absence of a state mandate to develop a 
set performance indicators for funding.. This movement has been extremely pervasive in the 
public sector with more than half of the states either using or planning to use various 
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performance measures to determine funding for institutions. (Brenda Albright, 1998, SHEEO, 
The transition from business as usual to flmding for results: State efforts to integrate 
performance measures in the higher education budgetary process). The presence of the state 
requirement has some advantages and also some disadvantages. Regardless of which side you are 
on, having a set of performance indicators tied to the budget process will move the culture of an 
institution to think a lot more about core data structures. As a private institution, the funding is 
based on revenue from student registrations and the expenditures are based on expenses for 
personnel. There is not the set of measures of outcomes that focus campus culture on quantitative 
and analytical reasoned empirical thought. Of course the good aspect is that we as a campus are 
not required by an external entity to use our resources to increase some indicator that we do not 
feel represents our mission and purpose. 

External Consultants: As part of the Title III and also in follow-up to the North Central 1997 
Self-Study we have had several consultants visit us and discuss many of the changes that we 
have made and are considering making. The specific recommendations have included the need to 
work on improving the quality and accessibility of our databases, the timeliness of our reports, 
and the integration of our services with the needs of our customers. These are the standard goals 
that are central to the continued improvement of an information and data management such as 
traditionally found in an Institutional Research Function. What is not so common, however, was 
the conclusion that the sheer tempo of implementation of change was causing the university to 
experience two basic themes. The first theme was that the range of activities was producing a 
broad range of individual initiatives and these initiatives caused the programs to have the 
potential of becoming silos to include the development of silos in the information infrastructure. 
The second issue was that there was a growing sense of sheer overwork. The change processes 
took a great deal of effort and were frequently seen as “additional work” by many of the 
involved individuals. This also resulted in the need to save energy by coordinating, 
consolidating, and integrating the changes into the operational process. Change needed to 
become status quo and be accomplished with a “business as usual” perspective. (Ewell 1999; 
Borden 2000) 

PeopleSoft: Primarily over the last year the university has begun to migrate its administrative 
information systems to PeopleSoft. This includes Student Administration which is going up at 
present. Finance which will be going up this Summer to Fall and Human Resources which will 
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also be going up this Summer to Fall. The implementation of this system has involved some 
forty or so functional and technical people directly and some several hundred individuals 
indirectly. It will create new measures, require re-coupling our reporting and data management 
out-flows, and provide some immediate opportunities to change the way we do things. Some of 
the changes will heavily impact both advising and also the involvement of the student and 
faculty in the learning process. The main impact at present is that there are very few resources to 
build information infrastructures for reporting and decision support. The transition to the next 
phase, which is due after Fall 2000, is helped because we are already seeing what some similar 
institutions can do that are a bit further along than we are with the data and reporting structures. 

Step n. Identifying the Program Portfolio and the Opportunity to Change 
As these initiatives continue to move forward, the array of information about university activities 
needs to change. As Peter Ewell emphasizes, "each institutional improvement strategy will be 
successful only to the degree that it is planned, coordinated, and evaluated in the context of good 
information" (Ewell, 1984). However, initiating a university-wide 'data improvement' project 
requires the support of numerous academic managers, managers who are increasingly skeptical 
of further change. The development of the portfolio concept and its application to the academic 
program becomes a way to build on existing information and integrate other strategic changes. 

In this way, the portfolio unifies and integrates ongoing university change initiatives rather than 
adding to their number. 

The concept of a program portfolio draws upon the idea of an institutional portfolio. Several 
years ago, six public urban institutions gathered together and concluded that they had a problem 
explaining themselves to their various constituents. Aided by a Pew Grant, the urban 
institutional portfolio project is designed to “enhance understanding, among both internal and 
external constituents, of the rapidly evolving roles and educational missions of urban public 
universities; and to improve the capacity of these universities to communicate, through the 
institutional portfolio, about their effectiveness in achieving those missions.” It was their feeling 
that the character of these institutions was substantially different from other types of institutions 
in their purpose, processes, and services. This created the need for them answer the central 
guiding questions; 

• “Who do these institutions serve and what are the expectations of their various 



stakeholders? 



• Through what kinds of programs, activities, and supports are these services provided? 

• In what kinds of circumstances and environments are services provided? 

• Toward what ends does the institution conduct its activities? 

• What are the outcomes for students, faculty, institution, and its communities? And 

• What is the quality of the processes and products of the universities?” 

These institutions felt that answering these questions and providing those answers to their 
various publics would represent a means to share information that supported program evaluation, 
institutional benchmarking, and internal planning and improvement efforts. This project 
continues with web based portfolios being developed, (see 
h ttp;Z/www, imir.tuPui.edu/po r tfolio/ProiSummarv.htm. and 
http://.vyww.o irP.Pdx.edu/portweb/index.htmV 

The idea of a program portfolio as a collection of information that can be used for reflection, 
advocacy, and measurement purposes also draws from the concept of student portfolios used for 
assessment at the classroom and program levels. Portfolios have gained popularity recently as a 
type of 'performance assessment,' or the process of using student activities or products, as 
opposed to tests or surveys, to evaluate students' knowledge, skills, and development. The 
student portfolio assessment method at the course level involves collecting samples of student 
work over time or within a course and a) rating or scoring the work and giving feedback to 
students; b) analyzing the samples for key traits, and/or b) having the student reflect on the 
quality of work (Palomba, p, 1 15-1 17) 

As with institutional portfolios, what makes a student portfolio useful for the purposes of 
assessment at the department level is the analysis of the portfolio to find evidence of and 
describe progress toward desired outcomes. "Although portfolios have a long history of use in 
evaluating the progress of individual students in several disciplines, their use for. programmatic 
assessment is relatively recent. To be useful for programmatic improvement, faculty need to 
have some agreement about how portfolios will be viewed collectively. That is, how will results 
from portfolios be summarized in a meaningful way?" (Palomba, p. 1 33) The need is to build a 
model with common dimensions and categories while supporting the individuality of the 
program/student. 
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The idea to create a program portfolio at DePaul also stems from the observations and 
recommendations of external consultants who have encouraged us to begin building an 
"information culture that will facilitate the university's strategic development, especially in the 
areas of student learning and civic engagement. Information management and institutional 
research can be used to direct institutional attention toward strategic goals and objectives" 
(Borden, 2000). As DePaul works toward that goal, the development of a program portfolio is 

A 

one step toward meeting the need for 'coordination and definition of aggregate information used 
in planning, management, and evaluation,' (Ewell 1998). 



Step ni. Defining the Structure of a Program Portfolio 

In developing a “Program Portfolio," the goal is to bring together information from a variety of 
different sources and provide it to managers and decision-makers in an integrated way. Our 
conceptual model for a program portfolio is similar to that for student or institutional portfolios. 
It is intended to be rigorous in methodology while flexible in scope. We have developed the 
following definition: 

The program portfolio is a set of quantitative and qualitative facts about a program that 
describes the program and its unique characteristics to internal and external constituents. 
It is intended to provide an overview description of the program and contains data and 
information in various summary forms. It is longitudinal and contains multiple facets 
about the program. Where appropriate the portfolio should explain the goals of the 
program, how the program fits into its larger context, what the program does and how it 
does it, and what the results are of what the program does. 

The concept of a program portfolio was developed to support a program's ability to reflect on 
itself in a manner that enhances learning and to support its ability to explain its goals, purposes, 
and accomplishments to others. The portfolio provides a lens for looking at progress toward 
various organizational goals and purposes. The portfolio is promulgated to be an organization’s 
alternative to creating a standard set of “performance indicators” that reflect “faculty 
productivity” to answer an external constituency’s demand for “accountability." It is similar in 
concept to the idea of a "balanced scorecard" for businesses outlined by Robert Kaplan and 
David P. Norton, where the company measures its performance according to four perspectives: 
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1) financial measures; 2) customer knowledge; 3) internal business processes; and 4) learning 
and growth (Harvard Business Review). 

While developing the conceptual model of a program portfolio, one of the earlier challenges has 
been to clarify what is meant by 'program' as the organizational structure for data collection, 
analysis, and interpretation. The initial intent of the program portfolio was to provide a structure 
for the academic department in the various colleges and schools. The structure of the Portfolio 
would be based on units below the College/School. A unit would have some of the following 
characteristics: 

• Has assigned FTE faculty/staff 

• Has budget authority 

• Produces instructional activity for a set of courses, degree(s), certificate programs, or 
required learning experiences. 

• Is of sufficient size (Students/StafETunding) to be its own organizational unit with a 
unique nature and historical stability. 

The program portfolio structure is best described if the academic unit has all of the four 
characteristics described above. These criteria do fit a large amount of our programmatic 
activity. However, some of the academic programs, such as Honors, Liberal Studies, and other 
interdisciplinary programs, do not have all of these characteristics. Furthermore other programs 
have only part of some of the characteristics, such as the First- Year Program, and will have some 
assigned personnel and a large number of personnel attached in a more temporaiy fashion. 
Finally, some of programs do not have budgets, assigned personnel, or full time staffing but do 
have acknowledged leaders and meetings. The effective use of the Portfolio for representing a 
program and, upon summary representing a College or School, will depend very heavily on the 
timely and accurate identification of personnel, funds, and activities to the program within the 
code structures of the University. This is especially true with the more temporary allocations. 

The initial steps in defining this structure of a program portfolio involved presenting the concept 
and asking the deans to support the project by clarifying the units reporting to them. This 
resulted in a set of academic activities nested within each school or college. Nested under these 
activities were the degrees for which they had primary responsibility. The effort was also made 
to include the subject courses for each of the academic activities. Finally, for many of the 
colleges, we listed centers, institutes, and other administrative areas. The person who directed the 
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activity was also included on the list. Needless to say, in each area there were a certain number 
of unique activities, degrees, and/or courses that were general under the school or college. There 
are also some administrative activities that are general under the university but no academic 
activities are general under the university. This organization was then put together with the 
budget structure and the organizational structure of the human resources activity. In the new 
PeopleSoft administrative information system, this common structure will be identical across all 
three activities giving us a common code structure. We are rather proud of this common structure 
and feel that it was worth the effort to create. 



Step rv. Defining the data elements of a program portfolio 

Given this structure, the next step was to clarify what belongs in a Program portfolio. Since the 
goal of the program portfolio at DePaul is to bring together a core of data and information from a 
variety of different sources and provide it to managers and decision-makers in an integrated and 
timely way, the initial types of data and information to be included for a traditional academic 
unit were identified: 

• Student Demographics such as gender, age, source of previous course taking, ACT, & 
ethnicity; 

• Student Activities such as grades, course taking patterns, degrees, service activities, & 
student progression; 

• Programmatic Activity such as instruction, class size, service projects, grants, location of 
activity & events; 

• Financial Information such as budget for types of expenditures and sources of funds; 

• Staffing Demographics such as gender, ethnicity, degree, & position category; 

• Faculty Activities such as courses taught, service provided, students advised, professional 
activities, and projects; 

• Program Characteristics such as number of faculty, staff, and students; and 

• Program Outcomes such as assessed student learning outcomes. 



These types of characteristics would come from institutional databases, from program files, from 
surveys, and from third-party data-bases such as ACT. One key criterion for including measures 
is the feasibility of obtaining the measure in a timely manner with sufficient relevance and 
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reliability. The identification of effort spent by a program in its various key activities such as 
research, service, and teaching is an example of this challenge. Several individuals have felt that 
this is a desirable way to describe the resources for the program but the current data structure 
delivers the measure too late and not in sufficient detail to be included at this time. The interest 
in the measure has caused us to look at ways to shorten the time required and to modify the 
measure to be easier to collect. 

Compiling these program-level data and measures in a timely manner in the program portfolio is 
a step toward providing consistent, useful information to support decision-making. This 
management information," as described by Ewell in Assessing Institutional Effectiveness, 1) 
should exist at a relatively high level of aggregation; 2) is most usefully presented and 
interpreted as trend data and therefore must be consistently collected; and 3) must be easily 
accessible and support a range of ad hoc studies. (Ewell, 1988). A primary benefit of collecting 
different types of program-level information fi'om multiple sources within DePaul and 
aggregating it in one place is that the portfolio would then contain much of the information and 
data that would aid in academic program review, strategic planning, assessment of student 
learning, and that would also required by various professional societies and agencies such as 
North Central. Therefore, when someone requires data and information on a program, they are 
generally available rather thaii having to be created for each such request. 



Step V. Clarifying the Customers and Their Needs. 

As the concept and model of a program portfolio evolves, it has been crucial to consider the 
context in which the information in the program portfolio will be collected and how the portfolio 
will be used. At DePaul, information comes fi'om and flows between various strategic initiatives- 
-especially the Strategic Plan, Academic Program Review, as well as ongoing strategic 
management activities. The information gathered then informs their progress. Thus, the 
program portfolio provides the opportunity to establish and routinely share key information 
among faculty, staff, and administrators who are involved in Academic Program Review, 
Management Decision-making, and Strategic Planning Activities. In fact, once the program 
portfolio s basic structure and elements had been initially defined, the next steps have involved 
bringing together key potential "user groups:" participants in the university's strategic planning 
process, those who have experienced the academic program review process, and those involved 
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in discussions on management information. This group's discussions have refined the beliefs 
about what and how to provide the data and information for the portfolio, and how the program 
portfolio would be used specifically. It was their suggestion that a first draft of the portfolio's 
core data be developed through the Academic Program Review (APR) Process, since APR 
provides a clear context for the use of a program portfolio. The Appendix lists the elements and 
sources of initial core data provided to the faculty review teams during the first cycles of 
Academic Program Review. 

Academic Program Review. The primary purpose of APR at DePaul is to support an evidence- 
based study and assessment of the unit's progress towards its key goals. Ideally, the program 
portfolio will provide that evidence in the form of program-level data and information about a 
program's students, faculty, instructional activity, etc. While the elements of the program 
portfolio itself are primarily descriptive, it is reflection upon that information and the analysis of 
those facts within the context of the program's goals and priorities which can turn the portfolio 
into a useful tool for academic program review. • Peter Ewell noted in his consulting visits to 
DePaul that "as other universities have found during their program review processes, using 
performance data to "discipline" the conversation focuses attention on concrete evidence and 
provides a common starting point for all units" (Ewell, 1998). By disciplining the program 
review conversations with evidence, the program portfolio can help faculty determine how the 
program is making progress toward its goals, and can assist the unit, the college, and the 
university in identifying and prioritizing action plans to improve academic quality. 

The decision to use the Academic Program Review process as the driver for establishing the core 
set of data was an important one, since the purpose of the data in this lens is to inform a 
reflective process. As in Boyer’s Scholarship of teaching, not only does it have to be relevant to 
the issues and engaging of the participants, and interesting, it must also be public for discussion, 
open to critique and evaluation and in a form that others can build on. (Hutchings and Shulman, 

1 999). This, for example, greatly reduced the ability to use various financial data since as a 
private institution these data are not in the public domain. 

The sequence of time and the interest of those involved in the APR process has created an 
interesting challenge. On the one hand, the APR process is a very positive force for the program 
portfolio as the program portfolio provides a means for the documentation of facts and 
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information about the program to support the reflection desired in the review process. At the 
same time, there is a need to make the Program Portfolio more than an annual program review 
document. It is also important that the program portfolio not be seen as a accumulation of the 
annual program reviews of the faculty as this would also tie it too closely with the administrative 
process. 

With that said, it should be noted that Academic Program Review is a relatively new change 
initiative in the university; if the program portfolio proves useful to this process, it may bring 
DePaul one step closer to the broader goal of building a culture where information is used on a 
regular basis to inform decisions about academic quality. Additionally, as program review 
evolves, so might the program portfolio evolve beyond just a collection of core data. Henry Ingle 
has noted that the construction and use of institutional portfolios should be guided by the 
institution's goals and should be subject to periodic review. He recommends that ”. . .the pieces 
of work or information included in a portfolio are purposefully selected to demonstrate progress 
toward a particular goal, a stated aim, or a predetermined set of benchmarks. ...This information 
is collected over a period of time, in a variety of modes, and is periodically reviewed to show the 
depth, breadth, and development of the accomplishment of the designated goal or objective. 
(Ingle in New Directions for Institutional Research, as cited by Borden in consulting report 
2000). Similarly, as programs begin to gather evidence themselves to track their progress toward 
key initiatives, they may begin to add to the portfolio with specific audiences and purposes in 
mind. In this way, the program portfolio can foster the type of information culture that can guide 
priorities and change initiatives at the program, college, and university level. 

Management decision-making. Beyond its role in focusing the program review self-study 
process, the program portfolio is also provides the opportunity to coordinate and report data that 
informs management decisions on an ongoing basis. The program portfolio is defined to support 
decision-makers in the continued improvement of the program; this in turn is heavily dependent 
on the strategic management process of the program. The strategic process of a program is to be 
aware of self, purpose, and capabilities, and to be aware of its environment. It then needs to chart 
a course considering relevant changes in the environment and to integrate these elements into a 
purposively sequence of intended events with consistent resources and appropriate modifications 
based on the context of the situation. The results need to be identified and refereed to interpret 
progress being made; and to adapt, alter, and adopt activities and the strategic model regarding 
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intentions and interpretations as time passes. In the development of the program portfolio, part of 
the process is to work within an environment that is shaping the strategic management and 
decision-making process of the programs. 



Strategic Planning. At the university level, strategic management decision-making is centered 
around the DePaul Vision 2006 Strategic Plan. Just as at the program level, the strategic process 
of the university is also to be aware of self, purpose, and capabilities, and to be aware of its 
environment. To the extent that the program portfolio can facilitate the sharing of program-level 
information within a standard set of categories, then the faculty committees and administrators 
who are involved with the university strategic plan might use that information to guide decisions 
and priorities for the university. For example, the use of the program portfolio for academic 
program review can not only discipline the self-study process within a unit, but it can give a 
structure and template of data and information that feeds back to the university strategic planning 
committees and initiatives. By analyzing a collection of program portfolios from units a given 
college or school, the university-level planiiers may begin to see patterns, trends, issues, themes, 
concerns, or other priorities emerging from the data in the portfolios. 

As noted earlier, to realize the potential of the program portfolio as a tool to inform 
review activities and strategic planning processes, DePaul ultimately must foster a culture where 
information is used to support management decision-making on an on-going basis. 

All too seldom do institutions make data the instruments of strategy in the follest sense— 
to gauge the capacity of an institution to folfill current commitments or pursue new 
opportunities, to understand external markets and the competition for new or existing 
programs or services, to analyze the opportunities for new ventures through 
collaboration among departments and centers, or to explore the prospects of even 
broader collaboration with other colleges and universities (Knight, 2000). 



Step VI. Looking Ahead to Next Steps in Deveioping and Using Program Portfolios 
Looking at the structure: 

With the linkage of management information to program review, there came a need to consider 
program portfolios as a strategy for Centers and Institutes as well as for the more traditional 
instructional activity, since centers and institutes will be reviewed. This has raised the issue of 
developing a set of categories of information for the program portfolio that are broader than the 
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traditional teaching, research, and service. One of the more interesting possibilities is the 
consideration of using Boyer’s four scholarships as the foundation. Boyer (1990) lists the 
following, restructured to be domains of activities : 

Domain of Discovery — MHiat has the program done in the investigation of issues and the 

creation of new knowledge and information within the various methodologies of its • 
paradigm? 

Domain of Application- What services has the program provided to help the community and the 
academy deal with its problems, challenges, and issues? 

Domain of Teaching-What have been the communal acts of sharing knowledge and bringing 
new knowledge to student and other communities that participate in the learning 
process? 

Domain of Integration- How has the program integrated its activities of the program with other 
key activities and concerns of stakeholders? 

Looking at the centers and at service-based learning has also convince us that these domains will 
need to fit within a broader context if we are to create a reflective context and content for 
developing and sharing information about programs. 

One possible context has been demonstrated by the work done on Outreach Programs by 
Michigan State University (1996) where the method for describing outreach used the criteria of: 
Significance: What is the importance of the issue and opportunities to be addressed and what are 
the goals/objectives of consequence? 

Context: How does the program fit with the University goals, values, and stakeholder interest? 
What is the fit with the professional expertise of the unit? What is the degree of 
collaboration and the appropriateness of methodological approaches and are there 
sufficient and appropriate use of resources? 

Scholarship: Here is where the four basic domains of activity can be fit into the consideration. 

What are the Knowledge resources, application, generation, and utilization? 
Impact:-What has been the impact on issues, institutions, and individuals. How sustainable is 

the program and what is its capacity to grow? What is its effect on relationships with key 
communities and how does it benefit the university? 

If we can apply the Balanced Score Card methodology (Kapland and Norton, 1 996) to the 
Michigan State categories, we will be able to focus more on the learning and growth perspective 
to include the competencies, infrastructure and climate necessary for sustaining quality. It can 
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also help make sure that we understand the linkages of the programs to the goals and strategies 
of the university. There seems to have been some success in the application of the scorecard to 
several types of academic departments (O’Neil, Bensimon, Diamond and Moore, 1999; Simione, 
Ammons and Rich, 1997). There is also some indication the movement to a Balanced Scorecard 
can be facilitated in the movement to the PeopleSoft data structure (Williamson, Johnson, and 
Harrison, 2000). 

This method uses a methodology that traces the events and activities of a program from start to 
finish in what is called a value chain. It then groups the activities and events into four basic 
areas: 

Customer/Stakeholder Perspective: How do customers see us? 

Internal Business Perspective: What must we excel at? 

Innovation and Learning Perspective: How can we continue to improve and create value for 
customers? 

Financial/Academic Management Perspective: How do we look to providers of financial 
resources and to the university leadership? 

The financial components of the traditional program description and the productivity of 
programs are both a bit problematic for us as a private university. At present, the first drafts of 
the more public core data will not have financial or productivity measures as such. These data 
are available on a restricted web site and are shared with deans and other senior administrators. 

Continuing the conversation: 

Although the development of a program portfolio is still in the initial stages at DePaul, 
significant progress has been made toward its refinement and implementation within the context 
of key change initiatives. The idea to establish and use a program portfolio has generated 
interest among faculty, administrators, and research staff, and it holds promise for addressing 
some of the multiple management information needs demanded by the rapid pace of change at 
DePaul. Already, working with the deans and associate deans to articulate a common structure 
for academic units has paved the way for ensuring that academic and management information 
needs are not subsumed by the transactional and operational data needs in the transition to 
PeopleSoft. So far, the conversations with coiisultants and with DePaul's information managers 
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and decision-makers has surfaced key issues and has led to a greater understandings of where the 
priorities should be as we work to improve the data and information systems at the university. 



Some immediate next steps that present themselves are as follows: 

1 . Share a draft of this paper with key groups within the university to foster discussions about 
the program portfolio 

2. Develop a program portfolio prototype or several different models based on the ideas 
generated in these discussions 

3. Pilot the use of the program portfolio and get feedback on its use from key groups 

As the program portfolio is further developed at DePaul, the key constituents will continue to 
discuss the concept, structure, and elemrats of the program portfolio so that it may be most 
useful for each program's management information activities. Its development will heavily 
invoke the participation of the Deans and other key individuals in refining the structure and a 
format that allows for a set of core information along with a set of organizational specific 
information that best provides for reflective opportunities for the organization. It is clear that the 
development of the program portfolio will evolve in discussion and in use, with consideration 
given to the unique context in which the portfolios will be used and for what specific purposes. 
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APPENDIX: List of Core Data Used In Academic Program Review 

Sample from the Department of English, DePaul University 



Notes on Sources and Categories of Data for English Department 
Student headcount 

Source: "DePaul University Fact File, 1994-1999." by the Office of Institutional Planning and.Research (OIPR). 
Unduplicated count of the unit's majors as of the fail quarters 1 994-1 999 after adds and drops {Undergrad, Graduate). 
Major code(s) used to calculate student head-counts: 200 (English), 205 (Writing). 

Degrees Awarded (for entire academic year) 

Source: OIPR Fact File, 1994-1999. 

Unduplicated count of the BA and MA degrees awarded by the unit to Its students for the entire academic year 
for the academic years 1 994-95 through 1 995-1 999. 

Student Demographics 

Source: OI PR/Registrar's quarterly enrollment statistics databases (1994-1999). 

Unduplicated counts of the unit's majors as of the fall quarter, after adds and drops (Undergraduate. Graduate). 

Minority category includes racial minorities and foreign students. 

"Part-time" designation is for students enrolled for less than 12 quarter hours. 

Enrollment Status of m^ors 

Source: Ol PR/Registrar's quarterly enrollment statistics databases (1994-1999). 

Status of the majors as of the foil quarters (1994-1999). 

"New transfers" are those undergraduates enrolled as transfers that quarter; this is NOT a count of the total number of 
majors who are transfer students. 

I^Relumlng" undergraduate students have re-enrolled as unit majors after 'stopping ouf 3 or more quarters. 
"Continulng"^undergraduate students are enrolled as unit majors from the previous quarter without 'stopping out.' 
"Upper level" undergraduate students have accumulated sufficient credit hours to count as juniors or seniors. 
Categories for graduate students are new graduate students, returning, and continuing (see above). 

Retention data 

Source: OIPR enrollment databases, foil quarters 1 994-2000. 

Unduplicated head-counts of students listed as unit majors in a unlf s degree program as of the fell quarter 
of the year listed. 

These retention/attrition rates are yearly "snapshots" of unit majors taken each foil. 

These retention/attrition rates are NOT longitudinal retention rates. 

Course Enrollments and Credit Hours 

Source: Office of the Registrar Quarterly Budgeted-Enrollment Comparison Reports, Autumn Quarters, 1995-1999 
Department codes referenced for unit course data: 23 (English), 90 (Writing), and 81 (Comparative Uterature) 

Total courses taught 

Total number of courses taught by the unit that quarter; does not include the sections with zero enrollment. 
Includes all types of courses: traditional, advisory/independent study, and foreign study courses. 

Total enrollments. Majors & Non-Majors 

Duplicated counts of students enrolled In the unit's courses: both majors and non-majors in unit courses. 
Average Class Size 

Total enrollment In all units courses divided by the number of courses taught in that quarter. 

All averages include all types of courses: traditional, advisory^ndependent study, foreign study. 

Total Course Credit Hours 

Generated by the enrollment of majors & non-majors enrolled In the unit's courses. 

Total Student Credit Hours 

Generated only by the unifs own majors taking courses In their home dept and elsewhere In the university. 
Faculty: Full- and Part-time 

Source: OIPR's Integrated Academic Information System, data as reported by the departments/colleges to OIPR. 

Note that data is not available for the two most recent years because It has not yet been received and/or aggregated. 
FTE Distributions are reported as an average for the unifs Full-time faculty only. 
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